Diversity of captured footage and accuracy requirements make the feature tracking problem very challenging. For instance, typical background footage exhibits drastic changes in lighting, motion blur, occlusions, and is usually corrupted with environment effects such as smoke or explosions. Tracking features on hero characters such as a human faces is equally challenging, especially near the eyes and lips, where the textures change continuously.
Existing commercial tracking packages can provide automatic approaches for identifying and tracking markers though a sequence. These tools may be very powerful in some situations, but can encounter strong temporal discontinuities such as occlusions or lighting flashes. Such “outlier” situations can be common in VFX work. Another class of trackers are single-marker systems, based on fast template matching algorithms. They can have a sophisticated user interface that provides manual controls to assist and guide the trackers through complex shots. However, because single-marker systems focus on individual features and not regions, they can have difficulty handling large-scale image motions such as scaling, rotation, blurring and large deformations of the image regions.